Scanning apparatus and method

ABSTRACT

A scanning apparatus and method for generating computer models of three-dimensional objects comprising means for scanning the object to capture data from a plurality of points on the surface of the object so that the scanning may capture data from two or more points simultaneously, sensing the position of the scanning means, generating intermediate data structures from the data, combining intermediate data structures to provide the model; display, and manually operating the scanning apparatus. The signal generated is structured light in the form of a stripe or an area from illumination sources such as a laser diode or bulbs which enable data for the position and color of the surface to be determined. The object may be on a turnable and may be viewed in real time as rendered polygons on a monitor as the object is scanned.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a United States national application corresponding to copendinginternational application PCT/GB96/01868, filed July 25, 1996, whichdesignates the United States, the benefit of the filing date of which ishereby claimed under 35 U.S.C. §371, which in turn claims the benefit ofBritish Application No. 9515311.0, filed July 26, 1995, the benefit ofthe filing date of which is hereby claimed under 35 U.S.C. 119.

FIELD OF THE INVENTION

This invention relates to an apparatus and method for scanning a threedimensional object.

BACKGROUND OF THE INVENTION

Real-world, three-dimensional objects whether with natural form (e.g.geographical, plant, human or animal-like) or man-imagined form (e.g.sculptures, reliefs, cars, boats, planes or consumer products) aredifficult to scan. This is because of features such as rapidly varyingsurface normals and surfaces for which a line of sight is difficultbecause it is partially obscured by other parts of the object.

Scanning machines—also known as digitizing machines—for scanning objectsor parts of objects can be categorised into two types: computernumerically controlled (CNC) and manually operated. A scanning machineincludes a unit that contains a sensing means commonly referred to as aprobe.

Objects or parts of objects can be scanned on CNC scanning machines witha number of computer numerically controlled (CNC) linear and rotatingmotor-driven axes. Different CNC machines can move/reorient the probe orthe object—or both—by a combination of translation and rotation aboutthese axes. Different machine designs are suited to different classes ofobjects. Probes can be temporarily or permanently attached to most typesof CNC machine tool or CNC coordinate measuring machine which can thenbe used for scanning. As examples, small and simple 3-axis CNC millingmachines may be used or large, complex 5-axis machines may be used. Thepoints captured by CNC machines are usually on a regular grid and therate varies from around 1 point per second up to around 20,000 pointsper second depending on the technology being used and the object beingscanned. The points from these scanning machines are accurate to theorder of 0.05 mm. CNC machines with probes scan by executing one or moreprograms that move the axes of the machine such that there is relativemotion between the probe and the object.

CNC machines are expensive, partly because of the incorporation ofmotors and the associated equipment for assuring rejection motion suchas linear guides and drive screws. Few CNC machines are flexible enoughso that the probe can be oriented in six degrees of freedom so as toscan the complete surface of a complex object. Even when a CNC machinehas six degrees of freedom, it is often not sufficiently flexible so asto position the probe to scan the complete surface of the object withoutcolliding with the object. When the object is a person or expensive, therisk of using a CNC machine may be unacceptable and there would be anecessity to make a machine to meet both the safety and scanningrequirements of the application. The programming of a CNC machine sothat the surface of the object is completely scanned without a collisionof the probe or machine with the object is often highly complex. Usuallythe design of the machine and the degrees of freedom inherent in thedesign and limitations in the probe design such as the standoff distanceduring scanning between the probe and the object, mean that it isimpossible to come up with a scanning strategy that will scan thecomplete surface of the object. It is common that the object has to bemanually picked up and replaced in a different position and ororientation one or more times during scanning. Each time that thisoccurs, the object has to be registered to a uniform coordinate systemsuch that the data from the different scans can be accurately combined.

Manually operated scanning machines can be categorised into three types:horizontal arm machines, multiply jointed arms and devices based onremote position sensing means.

Manually driven, horizontal arm measuring machines usually have threeorthogonal axes and are usually based on a travelling column design.These machines are usually quite large with the bed being at floor levelso large items such as cars can easily be moved onto and off them. Oftenmotors can be engaged on one or more axes to aid the manual movement ofthe machine. The probe is normally mounted at a fixed orientation on theend of the horizontal arm. This orientation may be changed and variousdevices may be attached between the end of the horizontal arm and theprobe to aid the changing of the orientation, most of these deviceshaving two axes. Horizontal arm machines have the disadvantage of notbeing able to easily orient the probe in six degrees of freedom. Thelimited flexibility in the design of a horizontal arm machine makes mostof the far side of the object unscannable.

Multiply jointed arms commonly comprise multiple linkages and areavailable for scanning complex objects. A multiply jointed arm typicallyhas 6 joint axes but may have more or less joint axes. At the end of themultiply jointed arm there is usually a tip reference point—such as asphere whose centre is the reference point or a cone ending in a point.Scanning is carried out by bringing the point or sphere into contactwith the object being scanned. The computer monitoring the multiplyjoined arm then measures the angles at all the joints of the multiplyjointed arm and calculates the position of that reference point inspace. The direction of the last link in the multiply jointed arm isalso calculated. Positions can typically be output continuously at arate of around 100 points per second, but the rate can be much more ormuch less. The accuracy is of the order of 0.1 to 0.5 mm. The pointsfrom the arm are usually sparse and unorganised. The sparseness and lackof organisation of the points makes it difficult to provide enoughinformation for constructing a computer model of the object that is ofacceptable quality. A multiply jointed arm with multiple linkages has alimited working volume. In general, if a larger working volume isrequired, the arms become very expensive, less accurate and tiring,difficult to operate. The limited working volume can be increased byleapfrogging in which the whole arm/base is moved to access anothervolume, but this requires a time consuming system of registering atleast 3 points each time the arm is moved and recombining the data setsfrom each arm position. Manufacturer's of multiplying jointed armsprovided pre-calibrated arms and test methods that the user may employto make sure that the arm is still calibrated to an acceptable accuracy.Such test methods use for example the standard tip reference point atthe end of the arm and a reference sphere or a ball-bar which is a rodwith two cylindrical cups that has a precise known distance between ahome ball and a end of arm ball. As the arm tip at the end of the ballbar is moved on the surface of spherical domain the arm recordspositions which are later compared to a perfect sphere and errorestimates for the arm are output.

Remote position sensing devices include hand-held devices that transmitor receive position information in a calibrated reference volume usingdifferent physical methods including electromagnetic pulses and soundwaves. A hand-held device may be connected to the rest of the system bymeans of a cable. These devices are prone to generating scanned pointswith very large errors and some devices cannot work when the objectbeing scanned has metallic components. They are less accurate thanmultiply jointed arms with accuracies of the order of 0.5 μm upwards.

There are three broad categories of scanning probe that could be mountedon the end of a multiply jointed scanning machine: point, stripe andarea probes. Point probes measure a single point at a time andtechnologies include mechanical contact methods and optical distancemeasurement methods. Stripe probes measure a number of points in a lineeither simultaneously or rapidly in a scanned sequence; the most commonstripe technology is laser stripe triangulation. Area probes measure atwo-dimensional array of points on a surface either simultaneously or ina scanned sequence; the most common technologies are interference fringeand multiple stripe projection. Some area methods require the device tobe held still for a few seconds during data capture. Stripe and areamethods have an in-built speed advantage over point methods as there areless motion of the probe relative to the object. There are differencesbetween the methods in terms of accuracy and cost but these do notgeneralise with category, for example a particular area technology maybe cheaper and more accurate than another point technology.

Means of capturing independent reference/feature points by contact arewell known and efficient. Structured light using stripe or area methodsis not good at capturing independent feature points because there is noway for the operator to align a known point on the object with a pointon the stripe or in the area.

Geometrical errors in the scanning process stem from many sources. CCDcameras can typically capture video at 25 frames per second. One majordisadvantage in normal use is that from any given demand for a framethere is a variability of 40 msecs until the start of capture of thatframe. If the probe is being moved at for example 100 mm/sec, this canlead to a geometrical error of 4 mm in the probe's data. The duration offrame capture depends upon the shutter speed e.g. {fraction (1/100 )}sec is 10 msecs. One major disadvantage in normal use is that if theprobe is being moved with a slow shutter speed, an additionalgeometrical error is created. An arm is typically connected to acomputer by a serial cable with arms typically generating positions at125 positions per second as they move; at this rate there is avariability of 8 msecs between when a position is needed at the computerand when it arrives. This can also introduce a geometrical error whenthe probe is moving. The total variabilities of the CCD camera and thearm can cause large aggregate errors.

There is a wide range of formats for 3D information in current use.These include the general categories: point formats, polygon formats andcomplex surface formats.

Point formats include: independent points, lines of points where a planeintersects with a surface, 2.5D areas of points which are commonly knownas range images which are single valued in Z and 3D point arrays whichare often generated by medical scanners. The point formats have manystandard representations including the Range Image Standard (RIS)resulting from the European ESPRIT Research & Development Project 6911,IGES and DXF published by AutoDesk Inc in the USA.

Polygon formats include polygons of different geometrical form. Polygonsmay be 3 or more sided and formats may include mixed numbers of sides oralways the same number of sides. Special cases such as Delaunaytriangulation can specify the positioning of vertices and the relativelengths of sides of polygons. Standard representations of polygonformats include STL published by 3D Systems Inc in the USA, IGES, OBJpublished by Wavefront Inc in the USA and DXF.

Complex surface formats include Bezier, NURBS, and COONS patches.Standard representations of complex standards formats include IGES,VDA-FS, SET, STEP and DXF.

The objective of scanning can be simply to gather a number ofthree-dimensional points on the surface of the object, or it may be tocreate a computer model in a format that is fuseful for the applicationin which the model is to be used. It is generally true that a cloud ofpoints alone is not much use in many applications and that morestructure is needed to make a computer model efficient to manipulate intypical applications such as visualisation, animation, morphing andsurface or solid modeling.

There are often benefits to be gained from reducing the size of thefiles associated with the model formats. Any file, whatever its formatcan be compressed using standard reversible utilities such asPKZIP/PKUNZIP from PKWare in the USA. With 3D point arrays, an octetformat can be used to reduce the size of the arrays that represent asurface; an octet format splits a cubic volume into eight, smaller cubesand only further subdivides cubes by eight if they contain information;an octet format is reversible. Moving from unstructured pointrepresentation to polygon or complex surface formats often produceslarge compressions but relies on approximations so the process is nearlyalways irreversible and it is also difficult to automate so as to givegood enough results. Chordal tolerancing is a commonly used method ofreducing the quantity of discrete points in a 2D or 3D polyline. As anintermediate data structure it has disadvantages in that: theintermediate data structure does not record the orientation of eachstripe, it does not record breaks in the data but assumes that all thepoints are connected by a surface and it does not record jumps in thedata such as those caused by occlusions.

Most scans today are carried out using a multiply jointed arm with a tipreference point. It is usual to firsf mark the object to be scanned witha permanent or temporary marking device such as an ink pen or scribe tocreate a polygonal network of splines. A single point is then scanned ateach network intersection. On the computer, the points are linkedtogether into a polygonal structure. The overall process (marking,scanning and linking) of creating a 3D polygonal model is at a typicalrate of 1 point (or vertex on the model) every 3 seconds. In someimplementations, the network is not marked on, but appears on a computerdisplay as each point is scanned; with this implementation, the networkis built up interactively. This method is suitable for models with arelatively small number of vertices i.e. hundreds and thousands. Themethod is very slow, requires skill, patience and concentration, isexpensive in human time particularly for large, detailed objects thatcan take 3 weeks to scan.

An alternative method of scanning with a multiply jointed arm andcontact tip reference point has often been tried in which independentpoints are rapidly captured without the aid of network. The points arethen input into a surfacing software package which then constructs apolygonal network between the points. However, the ‘polygonisation’ ofunorganised data points is usually very slow and speed decreasessignificantly as the number of points increases. The results are usuallyso poor as to be unacceptable. There is usually a significant amount ofhand editing of the data required.

Where a CNC scanning machine is used, the intermediate data structuresare usually range images. A number of unregistered range images may beregistered, polygonised and integrated together. The raw data is anumber of range images of an object: typically from 5 to 20 in number,with each one either being a cylindrical or a linear range image. Theprocess is not automatic and requires a combination of operator guidanceand automated execution of algorithms. The operator first tries to align(i.e. register) the range images to each other on the computer using agraphics display. This process is not accurate and is followed by anautomatic least squares fitting process that attempts to adjust theposition and orientation of each range image such that they fit togetheras well as possible; this process is lengthy often taking hours on apowerful computer. Each range image is then independently polygonisedinto a network of 2.5D triangular polygons. Finally, the networks oftriangular polygons are integrated together. The output is a single, 3Dpolygon dataset. The process is expensive both in terms of capitalequipment cost and people time. It can take up to two years to becomeskilled enough to scan objects to produce good enough models. It canwork and produce good results for detailed objects.

For smooth objects, where the objective is to create complex surfaceformats, a coordinate measuring machine with a contact tip referencepoint is commonly used. It is usual to mark up the object with thedesired surface patch boundaries by using a marking device such as a penor a scribe. These patch boundaries are then hand digitised with thecontact point probe. The software package then generates a CNC scanningprogram that automatically takes more points along the boundaries andinside the patches. The software then automatically generates a firstattempt at the surface model. This method is used because it is quickerand easier for the operator to define patch boundaries that will lead toa surface model with the desired structure before scanning than todefine the patch boundaries after scanning using a software package on acomputer with a display showing the scanned points. It can take severaldays and often weeks to create patch boundaries which are usuallysplines and then create the patches and then trim the patches to form asurface model by using only the scanned points and a computer.

Scanned data points have been displayed in real-time. The display ofpoints has the disadvantages of easily becoming confusing to interpretand also that the observer does not know when parts of the object'ssurface have been missed during scanning.

SUMMARY OF THE INVENTION

According to the present invention, there is provided a scanningapparatus for scanning an object to provide a computer model thereof,comprising means for scanning the object to capture data from aplurality of points on the surface of the object where the scanningmeans captures data from two or more points simultaneously; means forgenerating intermediate data structures therefrom; means for combiningthe intermediate data structures to provide the model; means for displayand means for manually operating the scanning apparatus.

The apparatus is most efficient in the quest to reduce the time and costof generating a computer model from a real world object by means ofscanning with both time and cost reductions of an order of magnitudeachieved over conventional techniques. The model is generatedautomatically from the intermediate data in a form that may beimmediately usable in a wide range of applications.

The scanning means may use structured light to more quickly scan thesurface of the object. The scanning means may also be operable to sensethe colour of the surface of the object, resulting in a model more likethe real world object.

Preferably the scanning means therein comprises means for generating asignal for scanning the object, signal detection means for detecting thesignal reflected from the object and means operable in response to thedetected signal to provide the data for the intermediate data structure.

The structured light is preferably projected as a plane of light suchthat a stripe is formed on a viewing plane that is situated normal tothe projection axis of the signal generating means and situated at theaverage stand-off distance from the signal generating means.

Alternatively the structured light may be projected such that a patternis formed on an area of a viewing plane that is situated normal to theprojection axis of the signal generating means and situated at theaverage stand-off distance from the signal generating means.

The signal generating means may be an illumination source such as alaser diode or one or more bulbs.

During scanning, the operator may see the surface he has scannedappearing in real-time as rendered polygons on the display such that hemay more easily scan the object. The operator may mount the object on aturntable and then he may scan from a seated position rather thanwalking around the object. The scanning means can be mounted on manydifferent types of manual machines, giving enhanced flexibility forobjects ranging in size from small to very large. The scanning means canbe mounted on a multiply jointed arm for accurate scanning. The scanningmeans may be a self-contained unit that contains a remote positionsensor and incorporates a display to give the most flexibility inscanning.

According to the invention there is also provided a method for scanningan object to provide a computer model thereof, comprising the followingsteps:

manually scanning the object with a signal by manual operation of asignal generating means:

detecting the reflected signal

generating intermediate data structures for the points;

combining the intermediate data structures to provide the model; and

displaying the data, wherein the data is captured from a plurality ofpoints on the surface of the object simultaneously.

According to a further aspect of this method of the invention the colourdata is also captured from the object and then mapped on to the model.

Preferably the data is displayed simultaneously as a plurality ofdisplay polygons.

The invention will now be described, by way of example only, withreference to the accompanying Figures, of which:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic representation of a scanning apparatus accordingto the invention;

FIG. 2 is a schematic perspective drawing of a probe;

FIG. 3a illustrates a first embodiment of the configuration of theoptical elements housed in a probe;

FIG. 3b illustrates a lamp configuration;

FIG. 3c illustrates an alternative to the lamp configuration of FIG. 3b;

FIG. 3d is a graph illustrating intensity as a function of distancealong the line A1-A2 of FIG. 3a;

FIG. 4 illustrates a second embodiment of the configuration of theoptical elements housed in a probe;

FIGS. 5a to 5 d illustrate a method of calibrating the colour of thescanning apparatus of FIG. 1;

FIG. 6 is a schematic block diagram illustrating the capture of colourand position data;

FIG. 7 is a schematic representation illustrating how the detection ofthe colour and position data is synchronised;

FIG. 8 is a schematic representation of the end of the multiply jointedarm of the apparatus of FIG. 1;

FIG. 9 is a schematic illustration of the turntable and multiply jointedarm of the apparatus of FIG. 1;

FIG. 10 illustrates the mounting of the probe on the multiply jointedarm;

FIG. 11 illustrates the alignment of the mount on the multiply jointedarm;

FIG. 12 illustrates the alignment of the probe on the multiply jointedarm;

FIG. 13 illustrates a linear range image;

FIGS. 14a, 14 b illustrates cylindrical range images;

FIG. 15 illustrates the range image placing method;

FIG. 16 illustrates the surface normal extension method;

FIG. 17 represents the structure of a single point in a range image;

FIG. 18 illustrates the representation of an object by three rangeimages;

FIG. 19 illustrates the range image updating method;

FIGS. 20a and 20 b illustrate first and second stripes captured on a CCDarray;

FIGS. 20c, 20 d, 20 e and 20 f illustrate the respective captured datapoints and strings of data points from the first and second stripes ofFIGS. 20a and 20 b;

FIG. 20g illustrates polygons generated from these strings;

FIG. 21 illustrates a probe mounted on a head and a head-up display;

FIG. 22 illustrates colour image mapping;

FIG. 23 illustrates the timing for position interpolation;

FIG. 24 illustrates the triggering of the arm position measurement;

FIG. 25 illustrates an object with marked lines thereon;

FIG. 26a illustrates a probe mounted on a multiply jointed arm which ismounted on a horizontal arm machine;

FIG. 26b illustrates two opposing horizontal arm machines;

FIG. 27 illustrates a human foot being scanned;

FIG. 28a illustrates stripe sections of a pipe network and panel;

FIG. 28b illustrates partial polygon models of a pipe network and panel;

FIG. 28c illustrates extrapolated polygon model of a pipe network;

FIG. 29a illustrates stripe scanning;

FIG. 29b illustrates area scanning;

DETAILED DESCRIPTION OF EMBODIMENTS

Referring now to FIG. 1, a scanning apparatus 100 comprises a multiplyjointed arm 1 with an arm control unit 2 and a probe 3. The control unit2, which includes a processing unit 10 is coupled to a computer orprocessing unit 4 and colour monitor 7. The probe 3 is also coupled to aprobe control unit 5 that is likewise coupled to the computer 4. Theintermediate data is displayed on the colour monitor 7 as renderedpolygons 13. The probe 3 provides a stripe 8, which is projected onto anobject 9 positioned on a turntable 14. The stripe 8 is in the form of aplane of light. Buttons 6 are also provided to control data capture. Acolour frame grabber 11 in the computer 4 is mounted on a standard bus12 and coupled to the probe 3.

The computer 4, probe control unit 5, arm control unit 2, buttons 6,colour frame grabber 11 and monitor 7 are provided separately, forexample, the computer 4 and monitor 7 may be a personal computer andVDU, although, for certain applications, it may be more convenient forone or all of them to be provided on the probe 3.

The multiply jointed arm 1 and the probe 3 are coupled to the computer 4by means of the control units 2, 5 discussed above. The computer 4receives information from the scanning stripe 8, theposition/orientation of the arm 1 in terms of X, Y, Z coordinates, withthe coordinates I,J,K of the surface normal of the probe 3 and colourdata if required.

Referring now to FIG. 2, an embodiment of the probe 3 for use withremote position sensor 261 is shown. The probe 3 is lightweight andresilient so as to withstand being knocked without losing itscalibration.

Referring now to FIG. 29a, the structured light is preferably projectedas a plane of light 364 such that a stripe 8 is formed on a viewingplane 360 that is situated normal to the projection axis 361 of thesignal generating means 362 and situated at the average stand-offdistance S from the signal generating means.

Referring now to FIG. 29b, the structured light may be projected suchthat a pattern 363 is formed on an area 365 of a viewing plane 360 thatis situated normal to the projection axis 361 of the signal generatingmeans 362 and situated at the average stand-off distance S from thesignal generating means. The patterns 363 in this example is a number ofstripes which may be of different colours.

Two alternative embodiments of the probe 3 are described. Now referringto FIGS. 3a; 3 b; 3 c; 3 d, one embodiment of the probe 3 is described.The probe 3 comprises a number of components mounted on a base plate 20.A stripe generator 22—for example containing a laser diode—provides thestripe 8 for projection onto the object 9 to be scanned. Typically, thelaser will be of Class 2 or less according to the CDRH 1040.11classification in the USA viz. less than 1 mW in power at 670 nmwavelength. The stripe 8 is nominally focused at some point P. A lensassembly 24 is used to focus the image onto a high resolution CCD camera25. The camera may be oriented at an angle satisfying the Scheimpflugcondition. An optical interference notch filter 26 is used toselectively image light of the wavelength of the stripe 8. A simple,glass cut-off filter 27 reduces ambient light within the probe.

Information on the colour of the surface of the object may be recordedin intensity scales or colour scales such as RGB.

An intensity scale estimate of the colour of the surface may be obtainedby recording the reflected light level of the stripe as it is imaged onthe high resolution CCD camera 25 at each point. A high level indicatesa light surface at that point which scatters much on the projected lightand a low level indicates a dark surface at that point that absorbs muchof the projected light. These indications may be false such as onspecular surface as known to a person skilled in the art.

A colour estimate of the colour of the surface can be obtained by meansof a colour camera 29 comprising a colour CCD array. Colour scanningrequires that the object is lit. Lighting may be by means of ambientlight from external light sources or by lamps situated on the probe.There are several disadvantages in using ambient light only for colourscanning: firstly, ambient light intensity varies over nearly everyobject in a standard environment such as a room with overhead lighting.Secondly, it can be a time consuming procedure to position a number oflights so as to evenly light the object. Thirdly, the probe itself cancast shadows onto the object.

Four lamps 28 a; 28 b; 28 c; 28 d ; are provided around the lens 31 ofthe camera 29 for illumination, or, a ring lamp 28′ could be used. Thisconfiguration is used to avoid any problems of shadowing. The lamps mayinclude respective back reflectors 32 a; 32 b; 32 c; 32 d whereappropriate. The lamps are set to give an average intensity of around80-150 Lux but the intensity could be much more or less and, during use,ambient light is reduced significantly below this level for example, bydimming or switching off overhead lights. This removes any effects fromvariations in ambient light. The lamps may be tilted with respect to thecamera axis to ensure that light of a more even intensity is projectedonto the object 9 at the average scanning stand-off distance. The lampsshould be small in size to obtain the least possible weight penaltyespecially if two or more lamps are used. To extend their life, they canbe operated at a lower voltage at certain time periods, for example,whilst preparing for the capture of each image. When the operatortriggers a colour image capture, the voltage to the lamps can bemomentarily increased to maximum. The lamps are only switched on forcolour capture. During the process of 3D capture the lamps are switchedoff where this will also have the added advantage of increasing thesignal to noise ratio. An access panel 35 can be provided over the lampsso that the lamps can be easily replaced without opening the probe andrisking losing its calibration. To improve results when scanning areflective surface, polarising material 34 is placed between the camera29/lamps 28 a; 28 b; 28 c; 28 d and the object 9. To reduce variationsin the projected light, diffusers 33 a; 33 b; 33 c; 33 d are placed inthe lightpath between each lamp and object 9, or, alternatively, thelamp glass or back reflectors are treated accordingly.

Referring now to FIG. 4, the second embodiment of the probe 3 isdescribed. As also preferable with the first embodiment, the base plateis provided with a removable two-piece cover 21 mounted on the baseplate 20 to define a housing for the components of the probe 3 toexclude ambient light and to protect the components. It may have ametallic interior coating to reduce electromagnetic emissions andsusceptibility. The cover is appropriate for both embodiments.

The probe 3 in the second embodiment can also capture the colour of thesurface of the object 9. The colour is captured using an optical systemthat is coplanar with the light stripe 8. The main advantage ofcoplanarity is that it provides the colour of each point directlywhereas non-coplanar systems such as in the first embodiment requireextensive post-processing computation to map the captured colour dataonto the captured 3D data. In non-coplanar systems the whereabouts ofthe strip in the colour camera is significantly variable due to thenon-alignment of the two cameras, leading to an embodiment which isoperated for scanning in two passes: 3D and colour instead of the onepass which is achievable from the first embodiment.

During use, with the strip generator 22 switched off and the lamp 28switched on a colour sample 32 of the object 9 is directed back alongthe direction of where the stripe 8 would be if the stripe generatorwere illuminated where it is reflected a half-silvered mirror 30 andfocused onto the colour camera 29 via a lens assembly 31.

In a probe 3 where the stripe generator 22 produces a white stripe, thecolour and position can be captured synchronously; the environment wouldneed to be dark in this case and lighting 28 would not be required.

A look-up method is provided for determining where to read the colourfrom the rectangular array of the colour camera 29 depending on theobject distance S at all points along the stripe 8. Now referring toFIG. 5, a colour camera look-up table 166 is prepared by viewing thestripe—when it is permanently illuminated—at a sufficient number ofpoints in a rectangular array covering the distance measuring depth andthe stripe measuring length. A fixed length, flat object 161 is one itemthat can be used for this purpose and it typically has a white surface.The flat object 161 is placed on a light stripe absorbent background162. The probe 3 and the flat object 161 move relative to each other inthe direction of the W axis such that stripes 163 are imaged. The colourcamera image 165 shows the imaged stripes 163 a collected in a region164 of the array. In the case of perfect coplanarity, the imaged stripeswill be superimposed (163 b in FIG. 5c). The look up table 166 is thenbuilt up so that, for a scanned point 167 on an object with coordinatesV1, W1 will have a colour image position Cx, Cy in the look-up table 166which stores an array of values of Cx, Cy for the V, W ranges. Duringscanning, the colour image is usually scanned before the stripemeasurement. The extent of the look-up table 166 can determine how muchof the colour image 165 need be stored whilst the points are beingcalculated ie the extents of region 164; this reduces the amount ofmemory needed in the computer 4 and the required bandwidth fortransferring colour information from the camera 29 into the computer 4,allowing the possible use of lower cost units such as the frame grabber11 located in the probe control unit 5 or in the computer 4 on a bus asdiscussed above. It is probably not worth the expense of building aperfectly co-planar system as a roughly co-planar system can becalibrated as described above to produce as effective a result.

Referring now to FIGS. 6 and 7, on co-planar systems, if the lightstripe 8 is of a colour other than white then it is difficult to capturethe object's surface position and colour at the same time. To overcomethis problem, the light stripe 8 and colour camera 29 can be switched onand off such that the colour is recorded shortly before or shortly afterthe position. Adjusting circuitry can be provided to change the exposuretimes of the colour camera 29 and the stripe generator 22 and to ensurethat the gap between the colour capture and the stripe captureminimised. To achieve this, a video synchronisation generator 60generates a synchronisation signal 61 of pulses at video camerarate—which using the CCIR format is 50 times per second. Thesynchronisation signal 61 is fed into the high resolution camera 25 andthe colour camera 29. The exposure time of the colour camera 29 can beset manually using a switch 63 or remotely with an electrical signal.The synchronisation signal 61 is also fed into a standard circuit 62which switches on the stripe generator 22 for a period of time after aninitial delay. The period of time that the stripe generator is on may beset manually using a control 64 and the delay between when thesynchronisation signal 61 is received and the stripe generator 22 isswitched on may also be set manually using a control 65. With referenceto FIG. 7, the synchronisation signal 61 is represented by the traceSYN. The illumination of the stripe generator 22 is represented by thetrace L. The exposure of the colour camera 29 is represented by thetrace C. The exposure of the high resolution camera 25 is represented bythe trace M. This is only one way by example of controlling co-planarprobes.

Referring now to FIG. 8, the probe 3 projects the stripe 8 so that themeasuring area starts before a tip reference point 51 provided at theend of the multiply jointed arm 1 and extends a further distance. Thetip reference point 51 may be the tip of a cone, a sphere or a rollingdevice such as a rolling wheel or ball or the point may be anythingsimilar providing a tip. In the embodiment described herein, the tipreference point 51 is the centre of a sphere 52. This tip referencepoint 51 enables the operator to scan hard objects by tracing the sphere52 along the object 9 in strips in contact with the object 9. For softobjects, the sphere 52 acts as a scanning guide and typical instructionsmight be to keep the tip reference point 51 about 20 mm from the object9 whilst scanning. In this way the probe 3 may be kept close enough tothe object 9 for the stripe 8 to be in the measuring area but withouttouching the soft object 9.

The stripe 8 typically starts 100 mm from the tip reference point 51 atthe end of the arm 1 but could be much closer or further away, and canbe used to measure objects lying between the two points W1 and W2 asmeasured from the end 55 of the probe 3. The idea method—from ausability point of view—is for the plane of the stripe 8 to be coaxialwith the axis 54 of the last section 50 of the arm 1, and this is thecase for a purpose designed arm and a hand-held probe. The probe 3 mayoften be retrofitted onto the arm 1 and because a mechanical arm has adiameter of typically 20-60 mm this presents an alignment problem. Inthis case, the plane of the stripe 8 is not coaxial but may be either ina plane 53 parallel to the arm end axis 54 or in a plane 53 a angled tothis axis 54 so as to cross the axis 54 at some point. By crossing theaxis 54 of the arm 1 somewhere in the measuring range, the ergonomics ofthe arm 1 can be enhanced because the light plane is in an easier to useposition. This crossover is typically somewhere between the tipreference point 51 and the end W2 of the measuring range.

Referring now to FIG. 9, the use of a manually rotated turntable hasseveral advantages. For a given arm size, larger objects can be scanned.The operator does not have to move round the object, this makes scanningphysically easier, more enjoyable and there is less a chance of eitherthe operator or the arm accidentally knocking in the object or of anyreference point being lost.

The position and coordinate system of the turntable 14 must be knownrelative to that of the arm 1. The tip reference point 51 can be placedin a locating cone or cup 202 in the table at a large radius. Points aretransmitted regularly by the arm control unit 2 and recorded on thecomputer 4 as the turnable 14 is manually rotated. Functions that fit aplane and a circle through these points provide complete position andorientation information on the turnable 14 in the arm coordinate system.

During scanning it is important to know the turntable angle. Theturntable may be designed to have precise mechanical resting positions206 a, 206 b, 206 c, 206 d, e.g. every fifteen degrees. These restingpositions 206 would be apparent from a pointer 208 indicating the angleon an attached scale 209 of three hundred and sixty degrees. Theoperator could type in the new angle into the computer each time theturntable was rotated. However the process of typing in an angle meansthe operator may have to put down the probe 3 and this slows downscanning and there is scope for an error to be made by the operator.

With an electrical connection 204 between a position sensor 203 on theturnable 14 and the computer such that the computer could know eitherprecisely or roughly the turntable angle, the process is faster and lesserror prone. If the sensor 20 is accurate—such as an encoder with, forexample, 10,000 lines—then the turntable 14 could be positioned at anyorientation and its angle known precisely. This allows for scanningwhilst rotating the turntable although care must be taken that dynamicsdo not lead to position errors or to the object moving relative to theturntable. If the sensor 203 is less accurate—such as apotentiometer—then the turntable 14 could also have precise mechanicalresting position 206. This gives the advantages of high accuracy andlower manufacturing cost. Each time the probe 3 captures data from theobject 9, the software must check for movement of the turntable 14; ifit has been moved then with a less accurate turntable sensor 203, probedata should be thrown away until the turntable 14 has stopped moving. Inall case the turntable 14 should be capable of being operated by onehand such that the probe does not have to be laid down. It is often thecase that an object on a turntable is scanned with regular incrementse.g. 8 scans every 45 degrees. To aid the operator in incrementing by Xdegrees different shaped and/or coloured icons could be placed every Xdegrees on the scale and other other regular intervals. Typicalintervals might be 45, 60, 90 degrees. With reference again to FIG. 2,this method can also be used with a probe 3 including one or more remoteposition sensors 261 with a tip reference point 51. The manual turntablemay be driven by a motor operable by means of hand controls.

Each time a probe 3 is mounted on the arm 1, if the mounting is notrepeatable to a high accuracy then the transformation in six degrees offreedom between the arm coordinate system X,Y,Z and the probe coordinatesystem U,V,W will have to be found.

A mounting device 210, 214 for the probe is illustrated in FIG. 10.Accurate and repeatable geometric positioning of the probe on the arm isrequired and this is provided by the mounting device 210, 214. Themounting device 210, 214 provides a standard mechanical interface whichmay preferably be used for all probes and arms and which is both smalland light and which is easy to use to mount and dismount the probe ontoand off the arm. The mounting device comprises a arm side mount 210which comprises a flat mating surface 211 with two precisely dimensionedprojections 212 located in precise positions on the mating surface 211.The mounting device also comprises a probe side mount 214 comprising aflat mating surface 215 and two precisely dimensioned recesses or holes216 corresponding to the two projections 212 in the arm side mount 210.It is essential that the geometric repeatability in position andorientation is very high.

A standard mounting device between any arm and any probe severaladvantages. When the arm has to be used without the probe then the probehas to be removed. If the mounting device is not repeatable then thesystem will require realignment before use each time the probe isremounted.

Typically a range of probes will be supplied with different weights,speeds, sizes and accuracies corresponding to different functions. Eachprobe can be supplied with alignment data relative to the datum of theprobe side 214 and in this way any probe may be attached to the armwithout the requirement of re-alignment. A user may also have one ormore different arms. In order to fit the same probe to two differentarms, the user need only acquire an extra adapter for the second armwhich would fit on to the arm and include the arm side mount 210 of themounting device 210, 214. The arm side mounting device 210 may beattached to any machine including multiply jointed arms, horizontal armsand orientation devices such as manual, 2-axis orientation devices.

To calculate the six degrees of freedom transformation between the armcoordinate system and the probe coordinate system one can either treatit as one transformation or as a multiplication of two transforms if theaccurately repeatable mount is considered as an intermediate referencepoint i.e. the transformation matrix Tap between the arm and the probecoordinate systems is equal to the transformation matrix Tam between thearm and the mount coordinate systems multiplied by the transformationmatrix Tmp between the mount and the probe coordinate systems:

Tap=(Tam)·(Tmp)

The transformation matrix Tam can be found in several ways. Nowreferring to FIG. 11, a particularly simple, cost effective andpractical method involves the use of a reference plate 220. Thereference plate 220 has three orthogonal flat surfaces 222 and amounting point 221 to which the arm side mount 210 can be attached in aprecisely known position relative to the orthogonal planes. Tam can becalculated using the following steps:

The reference plate 220 is fixed so that it cannot move relative to thearm coordinate system,

The arm side mount 210 is fixed rigidly onto the arm 1 (if it is notalready present), without the probe attached.

The three orthogonal planes 222 of the plate 220 are measured by the tipreference point 51 on the arm such as to fully define the position andorientation of the reference plate,

The arm mount is then mated with the mounting point 221 on the referenceplate 220,

The arm position and orientation is recorded,

The transformation matrix Tam is then calculated from the known geometryof the reference plate 220 and the measurements from the previous steps.

The above method can be encapsulated in the main scanning softwareprovided with the scanning system or in a separate program. This has theadvantage that much time is saved over an alternative of the usecalculating Tam manually from arm positions output by the armmanufacturer's software and manually inputting the resulting Tam intothe main scanning system software.

The probe side mount 214 is integral to the probe and does not moverelative to the probe coordinate system. The transformation matrix Tmpis provided by the probe supplier with the calibration data for theprobe.

The direct calculation of Tap using the arm and probe coordinate systemsbut without involving an intermediate mount can be carried out in manyways. Most of the ways involve using the probe mounted on the arm tocapture data from one or more geometrical objects. The problem hasproven to be very difficult since many of the standard methods produceinaccurate results in either the orientation or position componentsoften due to inherent instabilities triggered by relatively smallerrors. One way is disclosed by example for s stripe probe:

Now referring to FIG. 12, the transformation matrix Tap is calculatedby:

1. Mounting the alignment calibration object with three orthogonal faces230 so that the three orthogonal flat surfaces 231, 232, 233 arereachable and accessible by the probe 3 mounted on an arm 1;

2. capturing at least three stripes from three orientations on the firstflat surface 231—the orientations need to be different enough to providestability to the mathematical algorithm (in practice, a variation of atleast 5 degrees between every two of the stripes is sufficient);

3. repeating this three or more stripes capture on a second flat surface232 and on a third flat surface 233; and

4. processing the data from the nine or more stripes in an iterativefashion to output Tap.

The handedness of the coordinate systems of the arm 1 and the probe 3would be known. The relationship between the normals of the surfaces onthe alignment calibration object 230 could be specified. One way ofdoing this is by labeling the three faces 231, 232, 233 and specifyingthe order in which the three faces must be scanned.

The main advantages of the above apparatus and its method of aligningthe probe are: firstly, that it involves a single alignment calibrationobject that is cheap to manufacture to the required geometricaltolerance and is relatively light and compact; secondly, that the methodis robust, simply to carry out from written instructions and quick;thirdly, the processing can be encapsulated in the main scanningsoftware provided with the scanning system or in a separate system or ina separate program; fourthly, there is no need to have any preliminarygeometric information about the orientation and position of the proberelative to the tip of the arm at the start of this method, for examplethe probe could be slung on the underside of the arm pointing backwardsand the method would work and fifthly, if the probe is knocked ordamaged such that Tmp changes but the calibration is still valid, thenthis method of alignment will still work.

In use scanning systems to provide data for 3D applications software,the need for specific 3D reference points in addition to 3D surfacesbecome apparent. Some applications for which 3D surfaces are requiredwhich also require 3D reference points are animations involving jointmovements where a joint is to be specified in the context of the 3Dmodel; in this case the joint can be quickly defined from one or more 3Dreference points. A new method of using the scanning system is to usethe probe 3 to scan the surface and to use the tip reference point 51 tocapture individual 3D points by contact. An alternative method is toproject a calibrated cross-hair onto the object and use an opticalmethod of picking up individuals points. This can be used in both stripeand area systems. The calibrated cross-hair is usually switched on justduring the period in which individual points are captured. There couldbe two modes: in the first mode individual points are captured each timea button is clicked and in the second mode a stream of individual pointsare captured from when a button is first pressed until it is pressedagain. The second mode is commonly used for tracing out importantfeature lines such as style lines or patch boundaries. In the case of astripe sensor instead of projecting a cross-hair it may only benecessary to project a second stripe at the same time as the mainstripe. The cross-hairs may be calibrated by the probe supplied using a3-axis computer controlled machine, a known calibration object andstandard image processing techniques.

The scanning apparatus 100 is operable to scan an object and therebygenerate a computer model of the object's surface using an intermediatedata structure for efficiently storing points on the surface of theobject during scanning, creating an instance of the intermediate datastructure for the particular object, controlling the storage of thescanned points in the intermediate data structures during scanning withan operator control system.

Three examples of these intermediate data structures may be points orencoded stripes or range images.

Points have the disadvantage of being unorganised and much informationobtained from the structure of the probe and the method of its use islost if the 3D data is reduced to points.

In the case of stripe probes, much information may be retained toimprove the speed and quality of construction of a model fromintermediate data if an encoded stripe intermediate data structure isused.

Such a structure stores data from one stripe at one time. The stripesare stored in the order of capture. The time of capture of each stripeis recorded. The orientation of the probe is recorded for each stripe.The raw data points from the stripe may be processed before storing inthe data structure to determine jump and break flags and to sample orchordally tolerance the raw data points to reduce the size of theintermediate data structure without losing any significant information.

In the case of area probes, the advantages of a range image as anintermediate data structure as well known. These advantages include adata structure that relates well to the area based data capture methodand the efficiency of storage in an image in which only Z values arestored.

An intermediate data structure can be used in which the surface of anobject is described by means of a finite number of linear andcylindrical range images that are to some extent characterised by theshape of the object.

A linear range image 70 is illustrated with reference to FIG. 13. Therange image 70 has a coordinate system U,V,W and a spacing of point dUin the U direction and dV in the V direction. The linear range image 70contains in its header definition, its relationship to the worldcoordinate system X, Y, Z i.e. the arm coordinate system. In thedisclosed invention, the linear range image 70 cannot store negative Wvalues.

Cylindrical range images 71,72 are described in FIGS. 14a; 14 b. Therange image has a coordinate system W,R,A where A is an angle. Thespacing of points is dW in the W direction and dA in the A orientation.The cylindrical range images 71,72 contain in their header definitions,their relationships to the world coordinate system X, Y, Z. In thedisclosed invention, the cylindrical range images 71,72 cannot storenegative R values. The direction +R and position R=0 of a cylindricalrange image defines whether the points stored are inside the range imageas in FIG. 14a or outside as in FIG. 14b.

Referring now to FIG. 15, the range image placing algorithm takes ascanned point and tries to place it into defined range images byprojecting a ray along the normal to the range image 105. If the pointhas a negative value in a range image, for example point 104, then it isnot stored in that range image. If the point is outside the extents ofthat range image it is not stored in the range image unless the rangeimage is extensible in which case the algorithm extends the range imagefar enough to place the point. If a point already exists in the rangeimage position in which the point is to be placed, then the two pointsare compared. If the distance between the two points in space is outsidea tolerance d such as the error of the scanner, then the nearest point102 is stored and the furthest point 101 rejected. If the two points arewithin the error of the scanner then their values are averaged and theaverage value is stored.

The range image placing algorithm is simple and quick but, it isindiscriminate, often placing points incorrectly in range images andrelying upon them being overwritten by a nearer point. If the rangeimage is very dense but populated with few values then up to half thepoints populated could be incorrect because the surface normal of thepoint is incorrect; this can restrict successful scanning to coarserange images.

The range image placing algorithm is improved upon with the surfacenormal extension. The range image placing algorithm does not have anestimate of the surface normal of the point to be placed; also, it doesnot take into account the orientation of the probe when the stripe iscaptured. To improve the range image placing the fact that most stripesare scanned in sequence and have near predecessor and near successorstripes is used. For example, as illustrated in FIG. 16, there are eightneighbouring points 116 on a stripe 114 and on its predecessor 113 andsuccessor 115, and these can be used to approximate the surface normalof a point P before it is placed in a range image. Three sequentiallyscanned stripes 113, 114, 115 are shown on the an object 111 andprojected onto a range image 112 as stripes 113 a, 114 a and 115 a. Thepoint P, with coordinates Xp, Yp, Zp on stripe 114 has eight nearneighbours 116 on the respective stripes 113, 114, 115 as describedabove, and an approximate surface normal Np with coordinates Ip, Jp, Kp.The probe orientation for stripe 114 is N_(S) with coordinate is, Js,Ks. By calculating the surface normals N_(S), N_(P), and N_(R) whereN_(R) is the normal of the range image 112, one is given a choice oftwo, opposite surface normals; the correct one is the one that can beseen from the probe 3 orientation—assuming that the changes in probeorientation for the three stripes are not significant to the surfacenormal direction. If the surface normal NP of a point P is found to befacing away from the surface normal N_(R) then the point is not placedon the range image. This surface normal extension eliminates themajority of incorrect point placements in range images. In a practicalimplementation, three stripes of points are buffered before the firststripe of points is placed in the range images. The normal extension ina modified form can also be used for the first and last stripes by usingthe two successive or two previous stripes. When the three stripes 113,114, 115 are nearly coincident, perhaps because the arm is moving tooslowly, then the accuracy of the surface normal estimate is low and thenormal cannot be used. A difference normal calculation can be made usingany neighbouring points already placed in the range image instead of theneighbouring stripes. A further, normal extension to the range imageplacing algorithm combines both the stripe and the range image data toprovide a better estimate of the surface normal. The calculationsinvolved in these normal extensions can provide a bottleneck to thescanning process. The bottle neck can be overcome by using only twostripes, less samples (5 instead of 9) or a faster computer.

A number of range images that are positioned in the object coordinatesystem must be defined. The ranges images have specific mathematicaldefinitions. Two basic types of range image are used: linear rangeimages have specific mathematical definitions. Two basic types of rangeimage are used: linear and cylindrical as discussed above. A range imagehas direction and a zero position. The range image can only store pointsthat are in front of its zero position. If there are two more surfacesof the object in line with a point in the range image, the surface thatis nearest to the range image's zero position is represented in therange image. A range image can be constrained in size or unconstrainedin size. The range image can be one image fixed density or comprise apatchwork of a number of adjoining images of different densities. Eachgrid position in the range image is single-valued. The range image willtypically use 4 bytes to store a depth value Z, from 1 to 4 bytes tostore the greyscale or colour value I and from 1 to 3 bytes to store theorientation N. This is illustrated with reference to FIG. 17 whichillustrates how a single point is represented. The 3 bytes suggested fororientation N will not permit a very accurate orientation to be stored.More bytes could be used but there is a trade-off between data storablesize, processing time for converting floating number orientationsto/from a compressed integer format and accuracy. Ranges will normallyrequire from 5 to 11 bytes to store each point depending on theoperator's requirements. For comparison, 20 bytes are typically requiredto store an ASCII X,Y,Z value.

Now referring to FIG. 18, it is possible to define—for a finite object 9of any shape—a finite number of range images of the above types whichfor all practical purposes enable any and every point on the externalsurface of the object 8 to be stored in one or more ranges 81, 82, 83.

With objects with deep external features such as the inside of an ear,it may not be possible or practical to scan all parts of the externalsurface, but it is possible to represent them theoretically.

The number and position of the range images used in the process are suchthat they are sufficient to be able to store enough of the surface ofthe object to enable a computer model of the desired accuracy and detailto be generated.

In a manual process, the number and position of all the range images maybe defined by the operator before scanning. Alternatively, just one maybe defined by the operator before scanning begins followed by thedefinition of others at any point during scanning. The operator has achoice of several strategies: he can define range images and scan rangeone at a time. He can define a number of range images and scansimultaneously. He can define some range images and scan followed bydefining more range images and then scanning. If a point is scanned thatdoes not fit on to any defined range image then it is rejected.Alternatively, such reject points could be automatically saved forplacing into any new range images that the operator may subsequentlydefine.

A typical number of range images varies from 1 to 20. Some range imagesneed only be very small in size, small enough to cover a part of theobject that is otherwide hidden from recording on other range images.The density of each range image can vary, for instance a large, smoothpart of the object does not need a high point density. But, a small,finely detailed ornament may require a high point density. Each rangeimage has a direction.

The operator may select the most suitable set of pre-defined rangeimages from a library of range image sets. He can then edit the set tosuit his object. Each new set is then stored in the library. A set canbe thought of as a set of templates. As an example, for a human formthere could be a range image set consisting of 5 cylindrical rangeimages for the limbs and trunk, together with 5 linear range images forthe top of the head/shoulders, hands and feet. For a car, onecylindrical range image for the car's body and two linear range imagesat each end of the car could be enough. It is important to note that theaxis of a cylindrical range image must lie within the object or part ofthe object being scanned.

A range image is manually defined by the operator by firstly selectingthe appropriate range image type: cylindrical or linear and secondly,placing the probe to give the desired position and orientation of therange image and selecting it using the operator control system. For acylindrical range image, the probe could be positioned to first give theposition and direction of the axis and then to give the maximum radius.

Now referring to FIG. 19, an inference method is provided for updatingrange images from the other registered range images is also a novelmethod. The inference method progresses through each array position inthe range image 121 that is to be updated. The inference algorithm canupdate positions that either have no value or have a value with asurface normal that is steeper than a given value or have a less steepvalue or any combination of these according to the operator'srequirements. If a position in the range image 121 is to be updated,then that position is projected as a normal ray 126 onto all the otherrange images 120, 125, one at a time. If the ray intersects with anotherrange image 120 then the local triangular surface element through whichthe ray first passes is located on the surface 123 and constructed. Thevalue 124 at the intersection of the ray 126 and the triangular element122 is then inferred and placed onto the range image being updated. Ifthe ray intersects several range images 120, 125, then the infraredvalues from the range images are averaged after outliers have beenremoved. Outliers are removed by using a tolerance such as the error ofthe scanner. The original value (if it exists) in the range image beingupdated 121, could be included in this outlier removal/averagingprocess.

The interference method is particularly used when an additional rangeimage is added at a late stage in the scanning process or if rangesimages are defined/scanned one at a time. The method enables surfaceareas that are nearly orthogonal to the range image i.e. are almostvertical walls, to be well defined from data stored in the other rangeimages. This provides a better set of points for carrying out thepolygonisation of one range image resulting in a more accurate polygonalnetwork and simplifying the polygonisation process.

The probe 3 provides data which is displayed on the display monitor 7 asa rendered polygonal surface 13 in real-time or with an acceptable delaysuch that the user can watch the display monitor 7 and use the feedbackof the rendered surface to guide his movement of the probe 3. Real-timeis defined in the context of visualisation as an operation reacting witha delay small enough to be acceptable to an operator in normal use. Theprobe 3 could be a stripe probe or an area probe. Where the probecaptures 3D and colour information, then the colour information can bemapped onto the 3D model to texture it and this is discussed below.

The surface to be displayed is calculated for stripe probes oneadditional stripe at a time. Referring now to FIGS. 20a to 20 g, as astripe 301 is captured it is converted using one of several commonlyused methods into a finite string 303 of 3D points 302 a, 302 b, andflags 304, 305 in the world coordinate system X,Y,Z using the previouslyobtained calibration and alignment data for the probe 3. The maximumnumber of points is usually equal to the number of rows 281 or thenumber of columns 282 in the CCD array 25 depending on the orientationof the CCD in the optical setup. This disclosure will refer to rows butit can equally apply to columns or any other way of organising the datarecorded by the camera. Where the stripe crosses a row the position onthe CCD array can usually be calculated to sub-pixel accuracy by one ofseveral commonly used methods. If there is no data for one or moreneighboring rows such as missing positions 302 e, 302 f (not shown) thena ‘break’ flag 304 is output into the string to indicate a break in thesurface recorded. If there is a significant jump discontinuity in rangeabove a maximum value which is appropriately set for the scanningresolution of the probe 3 such as between 302 j and 302 k then a ‘jump’flag 205 is output into the string of 3D points to indicate either avertical wall relative to the probe orientation or an occluded surface.The string 303 is filtered to reduce the number of points whilsteffectively transmitting most of the information. The object offiltering is to reduce the amount of data processed for surfacerendering and hence increase the speed of the rendering process with aminimal degradation in the quality of the surface. The first method offiltering is to skip some of the stripes. Another method is to samplethe rows e.g. take every nth row. A third method is to chordal toleranceall the points in the stripe and discard the points that are surplus andwithin tolerance. Where computer speed is limited, the first and secondfiltering methods are preferred because of their simplicity and becausethe resulting regular grid of points produces regular polygons that lookgood on the display as opposed to long thin polygons that might resultfrom a chordal tolerancing process that can have rapidly changingsurface normals if the data points are slightly noisy due toinaccuracies in the probe and the arm and may present an unattractiveorange peel effect on the display. The same process is repeated for asecond stripe 306 capturing data points 307 resulting in a second string308 of 3D values 307 a, 307 b etc. and flags. A surface comprisingtriangular or quad polygons is then constructed between the two strings303 and 308 resulting in a string of polygons 309. The string ofpolygons are then displayed by a renderer. The renderer may or may nottake into account the previous polygons displayed, the viewpoint andlighting model.

If colour has been recorded for a polygon then the colour informationcan be mapped onto the polygon. The precise mapping algorithm depends onthe format of the raw colour information which depends on the design ofthe probe. The raw colour information may comprise point, line or areasamples. The raw colour information may be adjusted before mapping usingcolorimetric calibration and intensity calibration data. During themapping process the colour information may be adjusted for the probe topolygon distance at point of colour capture and polygon orientation toprobe at point of capture. The basis for the adjustments is a set ofcalibration procedures carried out for each individual probe.

The viewpoint for the surface displayed can have a constant position,zoom and orientation in the world coordinate system of the object suchthat as the probe is moved, the surface displayed increases where thedata is captured; the viewpoint is set before scanning starts eitherwith an input device such as buttons on the arm, footpedals, a mouse andkeyboard or using the probe to determine the viewpoint. Alternatively,the viewpoint can have a constant position, zoom and orientation in theprobe coordinate system such that as the probe moves, the surface iscompletely rerendered at regular intervals, each time with the newsurface displayed where the data has been captured with the regularintervals being at an acceptable real-time rate such as 25 displays persecond or less often. Alternatively, the viewpoint can have a constantposition, zoom and orientation in the world coordinate system where thesurface displayed increases where the data is captured that iscompletely updated to that of the probe coordinate system on operatordemand such as by the depressing of a button or footpedal or at regulartime intervals such as every 10 seconds. The different methods forupdating the viewpoint provide different advantages depending on thesize and type of the object being scanned and the speed of the computerin recalculating the surface display from a different viewpoint.

Referring again to FIG. 2, the display 7 can be mounted on the probe 3such that the rendered surface 13 or other image displayed moves withthe probe movement. Referring now to FIG. 21, the display 7 could bemounted in front of the operator's eyes as part of a head-up display 271which may or may not also allow the operator 270 to see his realenvironment as well as the rendered surface 13 or it can be mountedelsewhere. In practical use it is found that the operator watches therendered surface 13 on the display 7 whilst scanning because this hasthe important advantage of ensuring that all the object is scanned. Insome applications such as scanning large objects or using a turntable torotate the object, watching the display 7 as a large display monitorsituated on a workbench can be advantageous. In other applications suchas scanning a spherical type object a display screen on the probe 3 isadvantageous because the operator moves with the probe. With sufficientquality in a head-up display and with no negative effects such asfeeling sick, a head-up display may be best for nearly all applicationsbecause it is feeding back most directly to the operator.

Referring again to FIG. 2, it is already technically possible tointegrate the display 7 with the probe 3 as for instance a colour LCDscreen is small, lightweight, real-time and flat whilst havingsufficient resolution to render the surface so that the operator can seewhat has and what has not been scanned. A display mounted on the probecould be tiltable in one or two axes relative to the probe. The abilityto tilt the display relative to the probe can give the operator improvedability to scan in spaces with poor visual access. Buttons 6 on theprobe can be used to navigate menus and select from menus.

As computing power becomes faster and more compact it will be possibleto encapsulate the computer 4 in the probe 3 as well as having thedisplay 7 mounted on the probe. The probe might have memory 262 whichcould be both dynamic memory and magnetic memory such as a CD-ROM ordigital video disk (DVD). The probe might have a local power source 260such as batteries. This would be the case with one or more remoteposition sensors 261 mounted inside the probe. Although one remoteposition sensor is sufficient, more accuracy is obtained by averagingthe positions coming from three or more remote position sensor. Anotherbenefit of three or more sensors is that when a spurious position isoutput by one or more sensors this can be detected and the data ignored.Detection of incorrect positions is by means of comparing the positionsoutput by the three sensors to their physical locations within the probeto see if the variation is larger than the combined, acceptable error ofthe sensors. Since remote position sensor technology is likely to remainmuch less accurate than multiply jointed arm technology, it ispreferably that probes with remote sensors use array scanning meansrather than stripe scanning means. With a single array scan all the datain the array (ie a range image) is accurately registered to each otherbut with stripes there are position errors been any two sequentialstripes. It is possible to use an integrated close points (ICP)algorithm on overlapping range images to substantially reduce the errorscaused by the remote position sensors but this is not possible withstripes.

A number of different technologies exist for area probes includingbinary stereo, photometric stereo, texture gradients, range from focus,range from motion, time of flight, Moire interferometric and patternedstructured light systems. The most common systems in use in industrialapplications are time of flight, Moire and patterned structured light.Different area probe technologies have different advantages anddisadvantages for manual scanning.

Time of flight systems use a modulated laser spot to measure a scene bythe phase shift between outgoing and reflected beams which isproportional to the range of the object point; a complete range image iscaptured by scanning the whole region of interest. For a small area thistechnique is advantageous since it is line of sight although theaccuracy is generally of the order of 1-2 mm unless multiplemeasurements are taken at each point thus reducing scanning speedsignificantly. It is thus too slow.

Moire systems use gratings in front of projection and viewing optics toproduce an interference pattern which varies according to local changesin height on the object. Absolute measurements and measurements accrossdiscontinuities are only possible by taking several measurements withdifferent grating configurations or from different project angles. Forrelative height measurement these systems offer high accuracy. It isthus too problematic obtain absolute measurements.

A depth from focus range area sensor has recently been demonstratedwhich allow the real time determination of range from pairs of singleimages from synchronised cameras albeit with the use of relativelycomplex hardware. It is thus too complex to use at this point in thedevelopment of the technology.

Referring now to FIG. 29b, patterned structured light systems come inmany families and rely on projection of a light pattern and viewingoff-axis from the projection angle. Synchronously scanned lasertriangulation probes can be raster scanned over a 2D area. A laserstripe triangulation line can be scanned in one direction to producearea measurements. Scanning can be mechanical or electronic. Multiplelaser or light stripes 363 can also be simultaneously projected over anobject to obtain the same effect as a scanned stripe but this has thedisadvantage that in a single image it is not possible to differentiatebetween the stripes. To overcome this problem a number of systems use agray-coded sequence of binary stripe patterns which solves the ambiguityproblem but the sensor should remain stationary during the captureprocess. An alternative solution is the projection of colour coded lightstripes which allows the unambiguous determination of range even withdepth discontinuities from a single image. Note that the simultaneoususe of a number of stripes is herein classified as an area technique andnot a stripe technique.

The simultaneous projection of colour coded light stripes overcomes thedisadvantages of the previously described systems and is the preferredarea embodiment of this invention. Each stripe is one colour. Eachcolour may be a discrete wavelength such as provided by a number ofdifferent laser diodes or a subset of a spectrum range of colourgenerated from a white light source. Either all the colours may beunique or a small number of colours may repeat; the repetition of asmall number of colours can lead to ambiguity if stripes of the samecolour are not sufficiently separated.

The probe encapsulation would have advantages in terms of cost reductionand complete flexibility in freedom of use because even cables may notbe required and the only limits would be the range and accuracy of theremote position sensor.

If an arm is being used as the position sensor, the probe with a displaymounted on it might receive its power along a cable which may follow thepath of the arm and the computer may be suited in the base of the armwhich would reduce the weight of the probe and reduce operator fatigue.

Referring again to FIG. 21, if a head-up display 271 is being used, theprobe 3 with one or more remote position sensors 261 could be mounted onthe operator's head 270 with fixing means 272 to produce a head-mountedscanning system 274. This would lead to hands free scanning, althoughsome method of navigating menus e.g. verbally with speech recognition bymeans of a microphone 273 or via buttons would be important in practice.It is likely that the stand-off from the probe to the object using ahead-mounted scanning system 274 would be quite large for example 250 mmbut it could be more or less.

There are several ways of automatically polygonising intermediate datato form a 3D polygonal model. Two ways are described: strippolygonisation and range image polygonisation.

The strip polygonisation of intermediate data to automatically create apolygonal model is described for a stripe scanner. The followingdescription is by means of an example and comprises the following steps:

1. Take the intermediate data in the order in which it is scannedincluding the probe orientation for each stripe. For a stripe probe thiswill typically consist of a number of neighbouring stripes withoccasional discontinuities such as when the scanning process is pausedor a turntable is turned or the direction of scanning is reversed. Theintermediate data is preferably in an encoded stripe form as describedabove.

2. Group the data into stripes of similar probe orientations and nodiscontinuities. An acceptable variation of the probe orientation in agroup of data may be ten degrees. The average normal for each set ofstripes is specified. A new group is started each time a discontinuityappears or when the probe orientation varies unacceptably.

3. If not already done in the intermediate data, filter the stripes ineach group using a chordal tolerancing routing to reduce the quantity ofpoints and maintain the positions of the break and jump flags.

4. Use a 2.5D polygonisation method to polygonise each group. This willresult in a number of 2.5D polygonal meshes. There may be holes in anyof the meshes. The method eliminates occluded surfaces behind surfacesresulting from variations in the probe orientation within the group.

5. Use a polygon mesh integration method such as an implicit surfacemethod to integrate the 2.5D polygonal meshes into a computer modelcomprising one or more 3D polygonal meshes.

6. If required, use the known base plane of the object specified duringthe scanning setup to automatically close the bottom of the model wherethe object could not be scanned because it was resting on a table orturntable.

7. If required, use a general closing function to automatically closeall holes in the model.

8. If required, use a smoothing function set such that features createdby known levels of inaccuracy in the 3D scanning process are smoothedout and features greater in size than the inaccuracy of the system aremaintained.

9. Convert the internal polygon format into an output file of a commonlyused polygon file format such as DXF.

The range image polygonisation of intermediate data to automaticallycreate a polygonal model is similar to strip polygonisation. Each rangeimage is effectively a group of stripes with the same surface normal.Steps 1 and 2 above are therefore not needed. There are two ways ofcarrying out the equivalent of step 3 above. Range image data may bechordal toleranced as a series of stripes as described in step 3 and thepolygonisation process continued with steps 4 to 9 as required. In thesecond way, given the greater structure of a range image over a group ofstripes, steps 3 and 4 may be combined and a range image tolerancingalgorithm combined with a 2.5D polygonisation algorithm and thepolygonisation process continued with steps 5 to 9 as required.

Area scanners usually output range images. In general, range imagepolygonisation is better suited to area scanners and strippolygonisation is better suited to stripe scanners. If the intermediatedata structure is range images then the range image polygonisation willwork whether each range image relates to a particular data captureinstant or is part of a defined range image structure that ischaracterised by the shape of the object.

The combining of colour data onto the 3D model is known as texturemapping.

Before raw colour data in the form of colour images can be texturemapped onto the 3D model, it must first be corrected by means of variouscalibrations.

An important calibration is the geometric calibration of the colourcamera and finding the alignment transform of the colour camera to thecalibrated 3D measurement system in the probe. Without thesecalibrations/alignments neighbouring colour samples when mapped togetherwill produce visible errors. The objective of these calibrations is toget the geometric errors much smaller than those of the arm accuracy.The first geometric calibration is to take out lens distortion; standardmeans are used for this based on imaging geometric objects of known sizeand extracting pixel coordinates using standard image processingtechniques. The second is to create the camera model; a simple pinholemodel can be used or a more complex model; standard means are used forthis based on imaging geometric objects of known size from differentdistances and extracting pixel coordinates using standard imageprocessing techniques. The third is generating the alignment transform;a method has been developed based on 3D and colour imaging geometricobjects of known size using the probe. For all three methods a 3-axiscomputer controlled machine is used to ensure precise distances. Theprobe engineering must be geometrically stable enough such that thistransform will only be recalculated rarely such as after the probe hasbeen dropped or damaged.

Much of the effect of distance from the probe to the object on recordedlight intensity can be calibrated out. A diffuse, flat, white surface isimaged normal to the camera asix at a number of different distances fromthe probe to the surface. The distances are chosen to cover the wholescanning range from closest point to furthest point. The variations inmean intensity recorded in the camera are used to calibrate the probewith distance. This calibration data is used to correct the colour datarecorded when scanning an object such that all colour data is correctedto a known distance equivalent.

Much of the effect of tilt of the surface from the camera axis on thecolour quality can be removed but the effectiveness of this depends onat least the surface reflectance for each colour. A diffuse, flat, whitesurface is imaged at various angles to the camera axis at a fixeddistance from the probe to the surface. The angles are chosen to thepoint at which there is significant deviation from the Lambertian model.The variations in mean intensity recorded in the camera are used tocalibrate the probe intensity with relative surface angle to the probe.This calibration data is used to correct the colour data recorded whenscanning an object such that all colour data is corrected to a normalequivalent.

A standard colorimetric calibration is carried out using referencecolours such as Macbeth charts which are mounted normal to the colourcamera axis at a known distance from the probe. Corrections are made toa commonly used colour standard such as the CIE. Individual pixels inthe camera may be colour and intensity corrected.

Some of the above calibrations vary little amongst probes manufacturedto the same design. This is probably due to tight manufacturingtolerances. The calibration information can be incorporated into thesoftware as for example constants or tables or equations for the probedesign. Others calibrations are carried out once on the setup of eachprobe after manufacture. Other calibrations could be carried out eachtime the scanning system is used, for example the scanning of a whitesurface at a known distance will set the lamp intensity relative to theintensity when the bulbs were new.

Referring now to FIG. 22, there are several methods of mapping colourimages 320 onto the 3D model 324 to form texture maps. Surface elementson the 3D model may be flat polygons or elements of a high-level surfaceform. A mapping method for colour images is:

1. Each colour image 320 is corrected using calibration and geometricdata.

2. For each surface element 321 the colour image whose normal 323 isclosest in orientation to the normal 322 of the surface element 321 isselected (the master image) and the texture map coordinates for thatsurface element go to the mapping of that surface element onto thatmaster image. The closest image normal is that of 320 a in this case.

3. The other colour images that map onto the surface element are thenprocessed. If the surface normal difference between the surface elementand a colour image is above a certain tolerance, then that image isignored. This is because the colour quality obtained in the imagedegrades significantly as the surface orientation of the object relativeto the image becomes very steep. The part of the master image on whichthe surface element maps is then improved by a weighted average of allthe colour image mapped parts. The basis of the weighting is the cosineof the difference in surface normal between the surface element and thecolour image.

The apparatus and methods disclosed above each singly produce animproved colour ‘copy’ of the 3D model and a significant commercialadvantage.

Ways of improving the scanning timing and consequently reducinggeometrical errors are disclosed.

Where no electrical triggering is possible, to reduce the inaccuracycaused by the time difference between the recording of the arm positionand the capturing of the fame, the following method is employed:

1. With reference now to FIG. 23, the arm position before the frame iscaptured B is recorded and the time t1 of this recorded.

2. A frame is requested.

3. When the frame has been captured C the time t2 is recorded. There isa known delay T/2 with little variability from the middle of the framecapture to this time t2 which is largely dependent on the shutter timeopen T.

4. The arm position after A is recorded and the time t3 of this isrecorded.

5. The arm position in the middle of the frame is estimated byinterpolating in six degrees of freedom between the two arm positionsB,A using the time (t2-T/2)at the middle of the frame capture as theinterpolation weighting between t1 and t3.

6. In the case of a long interrupt if the difference between t1 and t3is significantly large then the data is deleted.

This interpolation method can increase the accuracy of a non-triggeredsystem by a large amount and is extremely significant in the quest toobtain geometrically accurate data.

In addition, the operating system under which the interpolation softwareruns may be set to prioritise the interpolation software is highpriority so that the introduction of delays due to other software beingexecuted is minimised. Even if another software function interrupts thisprocess, the validity of the process is not impaired unless theinterrupting process is of extraordinarily long duration. Prioritisationis not essential but will contribute to reduced timing error whereprioritising is available in the operating system.

In the case triggering is possible, there are many methods of carryingit out. One method is with reference now to FIG. 24 that thesynchronisation signal 240 from a CCD camera 25 is stripped off byelectronic circuitry 241 and a relay 242 is used to generate a series oftrigger pulses 243 to the arm computer 2. This has the advantage ofeliminating both the arm and camera variabilities and increasing theaccuracy of the scanning as much as possible for a given arm and camera.

The operator interface means alone—not including the standard computermeans such as mouse and keyboard—can be used to control the scanning andcomputer model generation process and the functionality of the optionsthat can be actuated. The operator interface means include means fornavigating menus such as: buttons, footpedals, joysticks, trackballs andthe position sensing means—arm or remove position sensor.

Using any of the above means, the operator can simply select therequired operations and operating parameters, which could include, forexample, being able to:

Setup scanning apparatus

select which position sensing device is being used ie arm

align the probe to the position sensing device; align the turntable

set the sampling of the points i.e. sampling step or chordal tolerance

set when data thrown away because arm is moving too fast

Data collection

prescan object to find out where it is

collect data points continuously whilst that option is selected forexample

collect one set of points such as a stripe section

collect sets of data points at pre-determined intervals of position

collect contact reference points

pause and re-start data collection

collect colour images

Process

generate polygonal or surface model from intermediate data

generate model in selected output format eg 3DS, OBJ

map colour images onto model

blend overlapping colour images on model

close holes in polygon mesh

slice polygon mesh

smooth polygon mesh

decimate polygon mesh

flip normals in polygon mesh

change datum and orientation of coordinate system

Edit

select/cut/past/delete points

select/cut/past/delete polygons

select/cut/past/delete colour images

Test

check the performance of the system by processing data from scanning asphere

check the performance of the system by processing data from scanning aflat surface

Display

display points in rendered colour according to depth

redraw the computer display from the position and orientation of theprobe

select the field of view of the redraw i.e. from zoom to wide angle

select the viewpoint from list of present viewpoints

display rendered data in one colour

display rendered data using the scanned colour data

display the computer model generated from polygons or complex surfaces

Model data

save the points/intermediate data/model onto a storage medium such as ahard disk

publish the intermediate data/computer model as an object such as anobject that may be automatically available to another software packageon the computer or over a network

load the points/intermediate data/model from a storage medium such as ahard disk

Range image

create a new linear range image using the position and orientation ofthe probe when the option is selected

create a new cylindrical range image using the position and orientationof the probe when the option is selected

select one of the defined range images from all the defined range images

change the density of that range image

delete the selected range images

delete all range images

select a set of range images from a library of range image sets; libraryrange image sets could be mathematically organised e.g. preciselyorthogonal to each other which may have advantages in some uses of thescanned data

add the selected library set to the currently defined range images

creating a new library set from the existing combination of rangeimages; in this way if several similar objects are to be scanned, theoptimum range image combination can be set for the first one andautomatically reused on the others

setting the selected library set as the default library set; in this wayfor instance a default library set of six range images that form a cubemay be used for many objects such that the process of range imagedefinition is not needed, making the total scanning process quicker

delete all current data points in all range images

delete all the data points in the selected range image only

display points from the selected range image only

display points from all range images

display points with different colours for each range image

update all range images from all the other range images by a process oftrying to fill in gaps or check entries in one range image by studyingthe others

update the selected range image from all the other range images by aninference process of trying to fill in gaps or check entries in onerange image by studying the others; this is particularly useful when anew range image is defined after a lot of scanning has taken place

constrain the size of a range image; this often done when a range imageis defined specially to capture a small part of the surface of theobject that is not covered by the other range images; this can be doneto save memory on a computer with limited memory and can also speed upthe whole process

choose and initiate an algorithm for automatically constructing a modelof polygons or complex surfaces from one range image

choose and initiate an algorithm for automatically constructing a modelof polygons or complex surfaces from all the range images

set the parameters such as the degree of accuracy by which an algorithmconstructs a model of polygons or complex surfaces

select an integration algorithm that combines the polygon model whichhave been generated from the range images.

select a predefined sequence of algorithms which automatically generatesa complete model of polygons or complex surfaces from a set of rangeimages

Complex surfaces can be created from marked surface patch boundaries.Referring now to FIG. 25, the object 130 is painted a uniform colour(ifnecessary) before marking the patch boundaries 131 by hand in anothercolour e.g. using a black marker pen on a white object. It is notimportant to mark these boundaries accurately as they usually lie awayfrom features such as edges or rapid changes in surface normal. Theobject is then scanned in using one of the methods disclosed. The colourinformation is then used to automatically generate the patch boundariesby means of an algorithm which separates out the points 132 lying on thepatch boundaries by means of a colour filter and then fits patchboundary lines such as splines 133 to these points. The edges may alsobe detected using a separate algorithm. The patch boundaries which havebeen automatically created from the scan can then be used to create thecomplex surface model. The main benefit of this method is that it iseasier to mark patch boundaries on the object than on the computer modelprior to the automatic creation of the complex surface model.

Referring now to FIG. 26a, an important implementation 333 of theinvention is disclosed in which the multiply jointed arm 1 is mounted onthe end of the horizontal arm measuring machine 330 for scanning a largeobject 331. The horizontal arm measuring machine 330 has a machinecontrol boss 332 which outputs the position of the machine to thecomputer 4. The arm control 2 and the probe 3 are also connected to thecomputer 4. This implementation makes the scanning of large objects moreprecise in that either a large arm or leapfrogging would be lessaccurate than a horizontal arm and simpler in that each time thehorizontal arm is moved the software takes it into account automaticallyrather than needing to reregister using a leapfrogging method. Inindustry firms who have large objects such as automotive manufacturersusually have horizontal arm machines so this makes the implementationparticularly attractive .

Referring now to FIG. 26b, firms who have large objects such asautomotive manufacturers often have two horizontal arm machines situatedopposing each other both of which can reference to the same objectcoordinate system. In this case the whole of the object may be scannedby scanning part of the object with the probe fitted to the firsthorizontal arm machine and the rest of the object with the probe fittedto the second horizontal arm machine.

This invention is a general 3D model making device and has wide rangingapplicability. The application industries for this invention include:design stylists who need to turn clay objects into computer modelsquickly and accurately; games developers and animators who need toconvert new characters into 3D data sets for animation; showmanufacturers who need to make custom shoes; automotive manufacturerswho need to model the actual cable and pipe runs in confined spaces;medical applications include radiotherapy and wound treatment.Altogether some 200 applications have been identified for thisinvention.

Referring now to FIG. 27, as an example of the applications for thescanning apparatus 100 in accordance with the invention, the scanningapparatus 100 can be used to scan a human foot 141 with full body weighton it on surfaces of different resilience is also disclosed. The outsideof the foot 141 is first scanned using the methods and devices disclosedabove with the required amount of body weight being exerted. The food141 is then removed and a second scan is carried out of the surface 142on which the foot 141 was pressed. The first scan is a positive. Thesecond scan is a negative. The surface normals of the second scan arethen reversed by means of a simple algorithm and the two scans combinedto give the positive shape of the foot. It is important that if adeformable material is used that it does not spring back. Such amaterial might be sand, clay or plaster. Materials of differentresilience may be appropriate for different applications. This method isalso appropriate when the foot is pressed onto the lower half of a shoewith the sides cut away.

There is a need by automobile manufacturers to identify the actual routeof pipes and cables in confined areas such as an engine department.Automobile manufacturers are trying to model in 3D CAD all aspects of acar. They need some way of scanning pipes and cables in the carreference system so that high level 3D models of the pipes and cablesare output that can be introduced into the CAD system for identifyingactual routeing and potential interferences. In the scanning of pipesand cables for instance in confined spaces if there is a problem withblack or shiny items not being scannable these can be first dusted witha white powder that is easily removed after scanning.

Referring now to FIG. 28, it is often better to scan a cable or pipe 341as a number of stripe sections 342 to 349 rather than a large number ofdensely spaced stripes. A stripe sensor can be activated in a first modeto take a single stripe section by the operator activating a button orfootpedal. In this way the operator can take a small number of sectionsto describe the path of the pipe using his expertise to decide when totake section. For instance where a pipe joins another pipe it may beappropriate to capture many more stripe sections 344 to 349. Also, wherethere is a feature such as a fixing on a pipe it may be appropriate tocapture very dense stripes. A second mode would be capturing stripesections as fast as the sensor can capture them and displaying them as asurface on the display. A third mode would be a mode in which theoperator specifies the distance between the sections e.g. 5 mm and thesystem automatically takes a stripe section every e.g. 5 mm that thestripe travels in 3D space. One method of determining this distance isto select the point at the average stand-off distance in the middle ofthe stripe i.e. the centre point of the measuring range and when thispoint has moved 5 mm to automatically capture another stripe section.When the operator is scanning pipes and cables, the operator controlsystem should support the simple switching between the three modes.

The intermediate data structure in which the stripe sections arecollated could be the standard stripe section structure 303 butincluding the changes in mode and the orientation of the probe for eachsection.

In scanning pipes and cables, panel sections along which the pipes andcables run are also captured 342 a, 342 d. Where there is no contactbetween the pipe and the panel, there is a jump or break in the stripesection. These can be flagged in the data structure with jump flags 305and break flags 304.

To be useful to an automobile manufacturer, a high level model should becreated and output from this data. A polygonisation or surfacing methodjoins the sections together and can handle the joining of pipes, panelsetc. The result is high level models 350 to 352. If more information isknown about the pipe or cable such as its section if it is constant orits form even if the form's dimensions change e.g. circular but varyingdiameter, the model 351 can be automatically expanded to 353.Alternatively two scanned sides of the same pipe can be automaticallyjoined. This gives the automobile manufacturer the high level model thathe needs.

As will be understood to persons skilled in the art, there are variousmodifications within the scope of the present invention. For example:The colour camera does not need to be included. A single camera could beutilised for both colour and position sensing. The filter in the probecould be a narrow band pass filter or a red high band pass filter asrequired. The system is adaptable to many types of model generation notjust those discussed herein. The data collected by the probe could beused for other applications, and could be stored for disseminationelsewhere, for example, by electronic mail. The probe can be a stripe oran area probe. The display can be mounted anywhere depending upon theapplication requirements.

What is claimed is:
 1. Scanning apparatus for scanning an object togenerate three-dimensional data, comprising: a scanner mounted on amultiply-jointed arm for movement by an operator to scan the object tocapture data from a plurality of points on the surface of the object,the scanner comprising: a light source operable to emit light onto theobject surface; and a light detector operable to detect light reflectedfrom the object surface by recording reflected light at recording timesdefined by a synchronization signal; a position calculator forcalculating the position of the multiply-jointed arm, and outputtingposition data defining the position in response to a trigger pulse; atrigger pulse generator for receiving the synchronization signal for thelight detector defining the recording times thereof, and, in responsethereto, generating and outputting trigger pulses to the positioncalculator to cause the position calculator to output position data foreach of at least some of the recordings by the light detector; and athree dimensional data generator for receiving recorded data output bythe light detector and associated position data output from the positioncalculator, and for processing the data to generate three-dimensionaldata relating to the object.
 2. Apparatus for use in scanning an object,comprising: a scanner for mounting on a multiply-jointed arm formovement by an operator to scan an object to capture data from aplurality of points on the surface of the object, comprising: a lightsource operable to emit light onto the object surface; and a lightdetector operable to detect light reflected from the object surface byrecording reflected lights at recording times defined by asynchronization signal; a trigger pulse generator for receiving thesynchronization signal for the light reflector defining the recordingtime thereof, and, in response thereto, generating and outputtingtrigger pulses for the position calculator of a multiply-jointed arm tocause the position calculator to output position data for each of atleast some of the recordings by the light detector; and athree-dimensional data generator for receiving recorded data output bythe light detector and associated position data output from the positioncalculator, and for processing the data to generate three-dimensionaldata relating to the object.
 3. Scanning apparatus for scanning anobject to generate three-dimensional data, comprising: a scanner mountedon a multiply-jointed arm for movement by an operator to scan the objectto capture data from points on the surface of the object, the scannercomprising: a light source operable to emit light onto the objectsurface; a light detector operable to detect light reflected from theobject surface; and a tip probe for touching the object surface atindividual points; a position calculator for calculating the position ofthe multiply-jointed arm, and outputting position data defining theposition; a controller for switching between a first data capture modeof the apparatus in which the apparatus is operable to capture data froma plurality of points on the surface of the object by irradiating theobject with light from the light source and detecting light reflectedfrom the object surface with the light detector while the tip proberemains present on the multiple-jointed arm, and a second data capturemode in which the apparatus is operable to capture data from individualpoints on the surface of the object touched by the tip probe while thelight source and light detector remain present on the multiple-jointedarm; and a three-dimensional data generator for receiving recorded dataoutput by the light detector and position data output from the positioncalculator, and for processing the data to generate three-dimensionaldata relating to the object.
 4. Apparatus for use in scanning an object,comprising: a scanner for mounting on a multiply-jointed arm formovement by an operator to scan an object to capture data from points onthe surface of the object, comprising: a light source operable to emitlight onto the object surface; a light detector operable to detect lightreflected from the object surface; and a tip probe for touching theobject surface at individual points; a controller for switching betweena first data capture mode of the apparatus in which the apparatus isoperable to capture data from a plurality of points on the surface ofthe object by irradiating the object with light from the light sourceand detecting light reflected from the object surface with the lightdetector while the tip probe remains present on the multiple-jointedarm, and a second data capture mode in which the apparatus is operableto capture data from individual points on the surface of the objecttouched by the tip probe while the light source and light detectorremain present on the multiple-jointed arm; and a three-dimensional datagenerator for receiving recorded data output by the light detector andassociated position data output from the position calculator of themultiply-jointed arm, and for processing the data to generatethree-dimensional data relating to the object.
 5. Scanning apparatusaccording to claim 3 or claim 4, wherein the light source and the lightdetector are arranged within a common housing and wherein the commoncommon housing is mountable on and de-mountable from themultiply-jointed arm without removing the tip probe.
 6. A method ofsynchronizing the recording of reflected light by a light detector andthe generation of data defining the position of the light detector in asystem for scanning an object to generate three-dimensional data, havinga scanner mounted on a multiply-jointed arm for movement by an operatorto scan the object to capture data from a plurality of points on thesurface of the object, the scanner comprising: a light source operableto emit light onto the object surface; and a light detector operable todetect light reflected from the object surface by recording reflectedlight at recording times defined by a synchronization signal; a positioncalculator for calculating the position of the multiply-jointed arm, andoutputting position data defining the position in response to a triggerpulse; and a three-dimensional data generator for receiving recordeddata output by the light detector and associated position data outputfrom the position calculator, and for processing the data to generatethree-dimensional data relating to the object; the method comprising:receiving the synchronization signal for the light detector defining therecording times thereof, and, in response thereto, generating andoutputting trigger pulses to the position calculator to cause theposition calculator to output position data for each of at least some ofthe recordings by the light detector.
 7. Scanning apparatus for scanningan object to generate a three-dimensional data, comprising: scanningmeans mounted on a multiply-jointed arm for movement by an operator toscan the object to capture data from a plurality of points on thesurface of the object, the scanning means comprising: a light sourceoperable to emit laser light onto the object surface; and a lightdetecting means for detecting light reflected from the object surface byrecording reflected light at recording times defined by asynchronization signal; position calculating means for calculating theposition of the multiply-jointed arm, and outputting position datadefining the position in response to a trigger pulse; trigger pulsegenerating means for receiving the synchronization signal for the lightdetecting means defining the recording times thereof, and, in responsethereto, generating and outputting trigger pulses to the positioncalculating means to cause the position calculating means to outputposition data for each of at least some of the recordings by the lightdetecting means; and three-dimensional data generating means forreceiving recorded data output by the light detecting means andassociated position data output from the position calculating means, andfor processing the data to generate three-dimensional data relating tothe object.
 8. Apparatus for use in scanning an object, comprising:scanning means for mounting on a multiply-jointed arm for movement by anoperator to scan an object to capture data from a plurality of points onthe surface of the object, comprising: a light source operable to emitlight onto the object surface; and light detecting means for detectinglight reflected from the object surface by recording reflected light atrecording times defined by a synchronization signal; trigger pulsegenerating means for receiving the synchronization signal for the lightdetecting means defining the recording times thereof, and, in responsethereto, generating and outputting trigger pulses for the positioncalculating means of a multiply-jointed arm to cause the positioncalculating means to output position data for each of at least some ofthe recordings by the light detecting means; and three-dimensional datagenerating means for receiving recorded data output by the lightdetecting means and associated position data output from the positioncalculating means, and for processing the data to generatethree-dimensional data relating to the object.
 9. Scanning apparatus forscanning an object to generate three-dimensional data, comprising:scanning means mounted on a multiply-jointed arm for movement by anoperator to scan the object to capture data from points on the surfaceof the object, the scanning means comprising: a light source operable toemit light onto the object surface; light detecting means for detectinglight reflected from the object surface; and a tip probe for touchingthe object surface at individual points; position calculating means forcalculating the position of the multiply-jointed arm, and outputtingposition data defining the position; control means for switching betweena first data capture mode of the apparatus in which the apparatus isoperable to capture data from a plurality of points on the surface ofthe object by irradiating the object with light from the light sourceand detecting light reflected from the object surface with the lightdetecting means while the tip probe remains present on themultiply-jointed arm, and a second data capture mode in which theapparatus is operable to capture data from individual points on thesurface of the object touched by the tip probe while the light sourceand light detecting means remain present on the multiply-jointed arm;and three-dimensional data generating means for receiving recorded dataoutput by the light detecting means and position data output from theposition calculating means, and for processing the data to generatethree-dimensional data relating to the object.
 10. Apparatus for use inscanning an object, comprising: scanning means for mounting on amultiply-jointed arm for movement by an operator to scan an object tocapture data from points on the surface of the object, comprising: alight source operable to emit light onto the object surface; a lightdetecting means for detecting light reflected from the object surface;and a tip probe for touching the object surface at individual points;control means for switching between a first data capture mode of theapparatus in which the apparatus is operable to capture data from aplurality of points on the surface of the object by irradiating theobject with light from the light source and detecting light reflectedfrom the object surface with the light detecting means while the tipprobe remains present on the multiply-jointed arm, and a second datacapture mode in which the apparatus is operable to capture data fromindividual points on the surface of the object touched by the tip probewhile the light source and light detecting means remain present on themultiply-jointed arm; and three-dimensional data generating means forreceiving recorded data output by the light detecting means andassociated position data output from the position calculating means ofthe multiply-jointed arm, and for processing the data to generatethree-dimensional data relating to the object.
 11. A method ofsynchronizing the recording of reflected light by light detecting meansand the generation of data defining the position of the light detectingmeans in a system for scanning an object to generate three-dimensionaldata, having: scanning means mounted on a multiply-jointed arm formovement by an operator to scan the object to capture data from aplurality of points on the surface of the object, the scanning meanscomprising: a light source operable to emit light onto the objectsurface; and light detecting means for detecting light reflected fromthe object surface by recording reflected light at recording timesdefined by a synchronization signal; position calculating means forcalculating the position of the multiply-jointed arm, and outputtingposition data defining the position in response to a trigger pulse; andthree-dimensional data generating means for receiving recorded dataoutput by the light detecting means and associated position data outputfrom the position calculating means, and for processing the data togenerate three-dimensional data relating to the object; the methodcomprising: receiving the synchronization signal for the light detectingmeans defining the recording times thereof, and, in response thereto,generating and outputting trigger pulses to the position calculatingmeans to cause the position calculating means to output position datafor each of at least some of the recordings by the light detectingmeans.